Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added input file sorting to many diagnostic to make output exactly reproducible #2710

Merged
merged 4 commits into from
Jul 8, 2022
Merged

Added input file sorting to many diagnostic to make output exactly reproducible #2710

merged 4 commits into from
Jul 8, 2022
+67 −30

Conversation

schlunma
Copy link
Contributor

@schlunma schlunma commented Jul 7, 2022 •
edited
Loading

Description

The exact output of many diagnostics depends on the order in which input files are read, which can vary between runs (e.g., 1D netcdf files with dataset coordinate whose order depends on the order of the input files). This makes the output non-reproducible and lets tools to check differences between the output fail (see #2704).

This PR adds sorting to many of these diagnostics to make them reproducible.

Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

New or updated recipe/diagnostic

To help with the number of pull requests:

@schlunma schlunma added this to the v2.6.0 milestone Jul 7, 2022
@schlunma schlunma requested a review from sloosvel July 7, 2022 15:17
@schlunma schlunma self-assigned this Jul 7, 2022
@schlunma
Copy link
Contributor Author

schlunma commented Jul 7, 2022

@sloosvel I will run all affected recipes overnight and post the output here so you can add these results to the table of recipes. That way we can also use these new runs as references and hopefully get less differences from the comparison tool in the future (together with #2709).

@schlunma
Copy link
Contributor Author

schlunma commented Jul 8, 2022

All recipes ran successful and the output ran fine.

@sloosvel could you please copy the output from here (/scratch/b/b309141/recipe_testing) to the results page mentioned here and have a brief look at this PR?

Thanks!!!

@schlunma schlunma marked this pull request as ready for review July 8, 2022 07:27
sloosvel
sloosvel approved these changes Jul 8, 2022
View reviewed changes
Copy link
Contributor

@sloosvel sloosvel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for re-running the recipes! I uploaded them to the results page, will regenerate the index.html now. Even though the comparison with 2.5 will still have differences because it was not sorted, the next comparison with respect to 2.6 should pass the tests for these diagnostics

@schlunma
Copy link
Contributor Author

schlunma commented Jul 8, 2022

Cheers, thanks Saskia!!

@schlunma schlunma merged commit 7c8b456 into main Jul 8, 2022
@schlunma schlunma deleted the fix_cox18nature branch July 8, 2022 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sloosvel sloosvel sloosvel approved these changes

Assignees

@schlunma schlunma

Labels
enhancement
Projects
None yet
Milestone
v2.6.0
Development

Successfully merging this pull request may close these issues.
2 participants
@schlunma @sloosvel